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Abstract — The capacity region of the Finite-State Multiple 
Access Channel (FS-MAC) with feedback that may be an 
arbitrary time-invariant function of the channel output samples 
is considered. We provided a sequence of inner and outer bounds 
for this region. These bounds are shown to coincide, and hence 
yield the capacity region, of FS-MACs where the state process is 
stationary and ergodic and not affected by the inputs, and for 
indecomposable FS-MAC when feedback is not allowed. Though 
the capacity region is 'multi-letter' in general, our results yield 
explicit conclusions when applied to specific scenarios of interest. 

I. Introduction 

The Multiple Access Channel (MAC) has received much 
attention in the literature. To put our contributions in context, 
we begin by briefly describing some of the key results in 
the area. The capacity region for the memoryless MAC was 
derived by Ahlswede in [1]. Cover and Leung derived an 
achievable region for a memoryless MAC with feedback in 
[2]. Ozarow derived the capacity of a memoryless Gaussian 
MAC with feedback in [3] 

In [4] [5], Kramer derived several capacity results for 
discrete memoryless networks with feedback. By using the 
idea of code-trees instead of code-words, Kramer derived 
a 'mulit-letter' expression for the capacity of the discrete i 1 
memoryless MAC. One of the main results we develop in the 
present paper extends Kramer's capacity result to the case of a 
stationary and ergodic Markov Finite-State MAC (FS-MAC), 
to be formally defined below. 

In [6], Han used the information-spectrum method in order 
to derive the capacity of a general MAC without feedback. 
Han also considered the additive mod-2 MAC, which we shall 
use here to illustrate the way in which our general results 
characterize special cases of interest. In particular, our results 
will imply that feedback does not increase the capacity region 
of the additive mod-2 MAC. 

In this work, we consider the capacity region of the Finite- 
State Multiple Access Channel (FS-MAC), with feedback that 
may be an arbitrary time-invariant function of the channel 
output samples. We characterize a sequence of inner and outer 
bounds for this region and show that it yields the capacity 
region, for the important subfamily of FS-MACs. 

Our derivation of the capacity region is rooted in the deriva- 
tion of the capacity of finite-state channels in Gallager's book 
[7, ch 4,5]. More recently, Lapidoth and Telatar [8] have used it 
in order to derive the capacity of a compound channel without 



feedback, where the compound channel consists of a family of 
finite-state channels. In particular, they have introduced into 
Gallager's proof the idea of concatenating codewords, which 
we extend here to concatenating code-trees. 

The paper is organized as follows. We concretely describe 
the communication model in Section [TT] In Section [TTTJ we 
introduce the causal conditioning, directed information and an 
important idea of sup/sub-additivity of regions. We state our 
main capacity results in Sections |IV] and [V] and we present a 
few applications of the capacity results in Section [VI] Because 
of space limitation we do not provide the proofs. The proofs, 
with the exceptions of Lemmas [3] and [5] and Theorem [4] can 
be found in the preprint [9]. 

II. Channel Model 

In this paper, we consider an FS-MAC (Finite State MAC) 
with a time invariant feedback as illustrated in Fig. Q] The 
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Fig. 1 . Channel with feedback that is a time invariant deterministic function 
of the output. 

MAC setting consists of two senders and one receiver. Each 
sender / G {1,2} chooses an index mi uniformly from 
the set {1, ...,2 n '} and independently of the other sender. 
The input to the channel from encoder I is denoted by 
{Xn, X12, X13, ...}, and the output of the channel is denoted 
by {Yl, Y2, Y3, ...}. The state at time i, i.e., Si £ S, takes 
values in a finite set of possible states. The channel is 
stationary and is characterized by a conditional probability 
P(yi, Si\xu,X2i, Sj-i) that satisfies 

P(Ui, Si\x\,x 2 , s 1 ' 1 ,y % ~ x ) = P(y i ,Si\x ll ,x 2 i,s i -i), (1) 
where the superscripts denote sequences in the following 
way: x\ = (xn, X12, xu), I £ {1,2}. We assume a 



communication with feedback zf, where the element zu is 
a time-invariant function of the output g/j. For example, zu 
could equal y\ (perfect feedback), or a quantized version of 
Ui, or null (no feedback). The encoders receive the feedback 
samples with one unit delay. 

A code with feedback consists of two encoding functions 
gi : {l,...,2 nRl } x Z^ 1 -> X[\ I = 1,2, where the kth 
coordinate of xf G X[ l is given by the function 

xik=gik(m u z^ 1 ), k=l,2,...,n, 1=1,2 (2) 

and a decoding function, 

g : y n -> {l,...,2 nRl } x {l,...,2 nR *}. (3) 

The average probability of error for ((2"" Rl , 2" fl2 , n) code is 
defined as 

^ = 2 n(R 1+ R 2) E Pr {.9^") ^ (^1, W 2 )\( Wl , W 2 ) Sent} 

(4) 

A rate (Ri, R 2 ) is said to be achievable for the MAC if there 
exists a sequence of ((2 nRl , 2 nR2 ), n) codes with Pe — * 
0. The capacity region of MAC is the closure of the set of 
achievable (Rx,R 2 ") rates. 

III. Preliminaries 

A. Causal conditioning and directed information 

Throughout this paper we use the causal conditioning 
notation (-||-). We denote the probability mass function (pmf) 
of Y N causally conditioned on X N ~ d , for some integer d > 0, 
as P(y N \\x N ~ d ) which is defined as 

N 

P(y N \\x N - d ) = l[P(y i \y i -\x i - d ), (5) 

i=l 

(if i — d < then x % ~ d is set to null). In particular, we 
extensively use the cases where d = 0, 1: 

N 

P(y N \\x N ) = l[P(y i \y i - 1 ,x i ) (6) 

i=l 
N 

Q{x N \\y N - 1 )=J[Q{x i \x i -\ y i - 1 ), ( ? > 

i=l 

where the letters Q and P are both used for denoting pmfs. 
The Directed information was defined by Massey in [10] as 

N 

I{X N ^Y N )^^I(X l -Y,\Y 1 ^ 1 ), (8) 
and in [4], Kramer introduced the notation 

N 

7(Xf -+Y N \\X%) =^2l{X[;Y t \Y l -\X l 2 ). (9) 

i=l 

Directed inofrmation has been widely used in the charac- 
terization of capacity of channels [4], [1 1]— [16], and rate 
distortion function [17], [18]. Throughout the proofs, we are 



using several properties of causal conditioning and directed 
information. We summarize them in the following lemma. 

Lemma 1 The following four properties .([T0t-(fT3l. hold for 

any discrete random vectors (Xf, X 2 , Y N ), 

P{x Y \y N \\x%) = -,x?). (10) 

|7(Xf -^Y N \\X 2 N )-I(X^ ->Y N \\X?,S)\ < H{S). (11) 

l{Q{x Y \\y N -%Q{x%\\ y N ^lP{y N \\x^x%)) 

= I(X? ->Y N \\X 2 N ), (12) 

where T(Q(xi), Q(x2), P(y\x%, x^f) denotes the functional 
7(X i; Y|X 2 ), i.e., 

I{Q(xi),Q(x 2 ), P(y\x!,x 2 )) = 

If there is no feedback, i.e., Q(x^ , x 2 \ ly^ -1 ) = 
Q(x?)Q(x£), then 

I{X^;Y N \X^) = I(X? -» Y N \\X?). (13) 

B. Sup/Sub-additivity, and Convergence of 2D regions 

In this subsection we define basic operations (summation 
and multiplication by scalar), convergence, sup-additivity and 
sub-additivity of 2D regions. Furthermore we show that the 
limit of a sup-additive sequence of regions converges to the 
union of all the regions, and the limit of a sub-additive and 
convex sequence 2D regions converges to the intersection of 
all the regions. 

Let A, B be sets in R 2 , i.e., A and B are sets of 2D vectors. 
The sum of two regions is denoted as A + B and defined as 

A + B = {a + b : a e A, b e B}, (14) 

and multiplication of a set A with a scalar c is defined as 

cA = {ca : a G A}. (15) 

A sequence {^4„}, n = 1,2,3,..., of 2D regions is said to 
converge to a region A, written A — lim A n if 

limsup„4„ = liminf An — A, (16) 

where 

liminf_4 n = {a : a = lima,,, a„ G A n } , 
lim sup An = {a : a = lima fe , a fe G A rik } , (17) 
and rife denotes an arbitrary increasing subsequence of the 
integers. Let us denote A = cl MJn>l -^™) A. = 

d(n„>x^). 

We say that a sequence {-4 n } n >i is bounded if sup{||a|| : 
a G .4} < 00 where | • 1 1 denotes a norm in R 2 . 

Lemma 2 Let A n , n = 1,2,..., be a bounded sequence of 
sets in R 2 that includes the origin, i.e., (0,0). If nA n is sup- 
additive, i.e., for all 71 > 1 and all N > n 

NA N 2 nAn + (N - n)A N - n (18) 



then 



lim An = A. 



Lemma 3 Let An, n = 1,2,..., be a sequence of convex, 
closed and bounded sets in R 2 . If nA n is sub-additive, i.e., 
for all n > 1 and all N > n 



then 



NA N Q nA n + (N - n)A N -n 
lim A n = A. 



code-trees (see Fig. This difference influences the 

(19) 

encoding scheme and the analysis. 
The following lemma establishes the sub-additivity of 

Lemma 4 (sup-additivity of TZ n . ) For any FS-MAC, the 
(20) sequence {TZ n } is sup-additive. Therefore, lim, woo TZ n exists, 
it is an achievable region, and it equals to TZ. 

(21) 

B. Outer Bound 



IV. FS-MAC WITH TIME-INVARIANT FEEDBACK 

A. Inner Bound 

Let TZ n denote the following region in IR 2 ^ (2D set of 
nonnegative real numbers): 

f R 1 < min SD -» y"||X 2 «, W, so) - ™, 

|J < i?2 < min S0 i/(X 2 " -> y" | |Xf , W, so) - (22) 
Q I Ri + R 2 < min S0 y((X 1> X 2 ) n -» y"|W, s ) - 



The following outer bound is proved using Fano's inequal- 
ity. 

Theorem 2 (Outer bound.) Let be an achievable 

pair for a FS-MAC with time invariant feedback, as shown 
in Fig. 

Q(x?\\z 
hold: 



log \S\ 



where the union is over the set of all input distributions 
of the form Q(w)Q(xY\\z^ 1 , w)Q(x^\\z^\ w). Having the 
auxiliary random variable W is equivalent to taking the convex 
hull of the region. Furthermore, the set of three-inequalities is 
equivalent to an intersection of three regions, and min So is 
equivalent to n So . Hence, an equivalent region is 
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) such that the following inequalities 
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-I(X[ l ^Y n \\X%)+e n , 
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-I(X^Y n \\X^)+e n , 
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Ri + R-2 
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-I((X 1 ,X 2 ) n ^Y n ) + e n , (24) 
n 



conv|Jf| 

Q s 



'Ri < $I(X?^Y n \\X$,s ) 
R2<^I(X^^Y n \\X^,s ) 
Ri + R2< y({X 1 ,X 2 ) n -> 



log 1 5 1 
log I -SI 



where e n goes to zero as n goes to infinity. 

This theorem implies that lim inf lZ n is an outer bound, where 
TZ n is defined as 



y n |s ) 



log 1 5 1 



where conv denotes the convex hull, and the input distribution 
is of the form Q{x r l\\z^ 1 )Q{x^\\z2~ l ). In general, the right 
hand side (RHS) of each of the three inequalities that define 
TZn can be negative. In such a case, we assume that the RHS 
is zero. 

Theorem 1 {Inner bound.) For any FS-MAC with time invari- 
ant feedback as shown in Fig. Q] and for any integer n > 1, 
the region TZ n is achievable. 

The proof is similar to the point-to-point FSC with time- 
invariant feedback, given in [14, Sec. V]. In the proof we 
use Gallager's techniques to analyze the error probability of 
a ML decoder of a randomly-generated code. There are two 
main differences compared to the point-to-point FSC: 

1) In the case of FSC, only one message is sent, and in the 
case of FS-MAC, two independent messages are sent. 
This requires that we analyze three different types of 
errors, and they yield three inequalities in the achievable 
region. 

2) For the FS-MAC case, we need to prove the achivebility 
for a set of input distributions while for the point-to- 
point channel it was enough to prove it only for the 
input distribution that achieves the maximum. Because 
of this difference, we introduce the idea of concatenating 



(23) (Ri<n-I(X{ 1 ^Y n \\X 2 

n n = |J \r x < i/(x 2 " Y n \\x\ 

Q (Ri+R2<^I((X 1 ,X 2 ) 



(25) 



Y n ), 



and 



the union is over input distributions of the form 

,71- 

2 



C. Capacity 

Based on the bounds above, we have the following capacity 
result. 

Theorem 3 For any FS-MAC of the form 

P(yi, Si\xi t i,X2,i,Si-i) = P(s i \s i -i)P(y i \xi t i,x 2 ,i, Sj_i), 

(26) 

where the state process Si is stationary and ergodic, the 
achievable region is lirrin^oo TZ n , and the limit exists. 

V. FS-MAC WITHOUT FEEDBACK 

The case where there is no feedback is a special case of 
deterministic time-invariant feedback in which z,; is null, and 
therefore the theorems in the previous section hold for the 
case of no feedback. Here we show additional results, which 
apply only for the case without feedback. The results include 
a sequence of upper bounds for all FS-MACs, and a capacity 
formula for indecomposable FS-MACs. 



codeword (case of no feedback) 



JUi = 



,%2 — 1 



code-tree (used in [14]) 
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.,■., = 




Zi-l = 1 
• — "-•(no feedback) 



Fig. 2. Illustration of coding scheme for setting without feedback, setting with feedback as used for point-to-point channel [14] and a code-tree that was 
created by concatenating smaller code-trees. In the case of no feedback each message is mapped to a codeword, and in the case of feedback each message is 
mapped to a code-tree. The third scheme is a code-tree of depth 4 created by concatenating two trees of depth 2. 



A. Outer bound 
Let us denote, 

Hn = 

f R 1 < maxp -» Y n \\X$, So) + ™, 

conv|J I R 2 < maxp i/(X 2 " -> F"||A7, S ) + ^1, 

« + R2 < maxp ^/((Xi.Xa)" - Y n \S Q ) + ^1 

where the union is over all input distributions of the form 
Q( x i )Q( X 2 )> anc l maxp denote a maximization over distri- 
bution of the form P(sq\x™ , x%)- The sup-additivity of {lZ n } 
is the key property for establishing the outer bound. 

Lemma 5 (sub-additivity of TZ„. ) For any FS-MAC, the 
sequence {7Z n } is sub-additive, i.e., 



A. Zero capacity 

The first conclusion is given in Theorem [6] 

Theorem 6 For the channel described in d26l ), where the state 
process Si is stationary and ergodic, if the capacity without 
feedback is zero, then it is also zero in the case that there is 
feedback. 

The proof of Theorem [6] is based on the fact that for any 
MAC 



(n + l)TZ n+ i C nR n + ITZi 



(27) 



Theorem 4 (Outer bound) For any FS-MAC and all n > 1, 
lZ n contains the capacity region. 

B. Capacity 

Theorem 5 {Capacity of FS-MAC without feedback.) For any 
indecomposable FS-MAC without feedback, 



lim TZ n = lim TZ n , 



(28) 



and therefore its capacity region is linin^oo lZ n , and the limit 
exists. 

VI. Applications 
In this section we use the capacity results in order to derive 
the following conclusions: 

1) For a stationary ergodic Markovian channels, the capac- 
ity region is zero, if and only if the capacity region with 
feedback is zero. 

2) For the additive mod-lA"! MAC, where the noise may 
have memory: 

a) feedback does not enlarge the capacity; 

b) source-channel coding separation holds for lossless 
reconstruction. 



max 



if and only if 



max I(X?,X% ^Y n ) = 0, 

Q(s?)Q(z£) 



(29) 



(30) 



and on the fact that for the family of channels that is mentioned 
in the theorem, the sequence lZ n is sup-additive. 

For the case of additive Gaussian MAC, one can deduce 
the result from the fact that feedback can at most double 
its capacity region [19]. Clearly, Theorem [6] also holds for 
the case of a stationary and ergodic FS -Markov point-to-point 
channel because a MAC is an extension of a point-to-point 
channel. However, it does not hold for the case of a broadcast 
channel. 

B. Additive mod-\X\ MAC 

In this section we consider the additive mod-| X\ MAC with 
and without feedback. The channel is described in Fig. [3] In 
the binary case, the channel is simply Y = X\_i © X 2 ,i © Vi, 
where Vi is the binary noise, possibly with memory, and © 
denotes addition mod-2. 

The following theorem is an extension of of Alajaj's result 
[20] to the additive MAC . 

Theorem 7 Feedback does not enlarge the capacity region 
of a discrete additive (mod-|,Y|) noise MAC. Moreover, the 
capacity region is given by 



M 
rn — 1 



<logq-H(V), 



(31) 



where H(V) is the entropy rate of the additive noise. 
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Fig. 3. Additive noise MAC with feedback. The random variables 
X± i, X% t i, Yi, Vi, i € Z + , are from a common alphabet, and they denote 
the input from sender 1,2, the output and the noise at time i, respectively. The 
output satisfies yi = x\ t i®X2,i®Vi where ® denotes addition mod-IA'l. The 
noise Vi, possibly with memory, is independent of the messages Wi, Wi. 

The theorem can be shown to hold for a larger family of 
MACs. The family includes all the MACs that can be rep- 
resented as multiplexer followed by a point-to-point channel. 
The main idea of the proof is that all three inequalities that 
defines the region lZ n are maximized by uniform and i.i.d 
distribution, even if feedback is allowed. 

C. Source-channel coding separation 

Cover, El-Gamal and Salehi [21] showed that, in general, 
the source channel coding separation does not hold for MACs 
even for a memoryless channel without feedback. However, 
for the case where the MAC is a additive mod-|A"|, and the 
goal is to reconstruct the sources losslessly, then it does hold. 

Theorem 8 (Source-channel coding theorem for a additive 
mod-\X\ MAC.) Let (Ui, Ui) n >\ be a finite alphabet, jointly 
stationary and ergodic pair of processes, and let the MAC 
channel be an additive mod- MAC with stationary and 
ergodic noise. Define P e (n) = Pr((17f, U$) + (U?,U?)), 
where U™ , U 2 are the reconstructed sources at the decoder. 

(direct part.) There exists a sequence of source-channel 
codes with P e (n) -> 0, if H(Ui,U 2 ) < log \X\ - H(V), where 
H(Ui,U2) is the entropy rate of the sources and H(V) is the 
entropy rate of the noise. 

(converse part.) If H(Ui,U 2 ) > log\X\ - H(V), then the 
probability of error is bounded away from zero, independent 
of the blocklength. 

It is interesting to notice that, even though the source-channel 
coding separation theorem holds when the reconstruction of 
the sources has to be lossless, the theorem does not hold when 
distortion is allowed. Such an example was shown by Nazar 
and Gastpar [22]. 

VII. Conclusions and Future Directions 

In this paper we have shown that directed information and 
causal conditioning emerge naturally in characterizing the ca- 
pacity region of FS-MACs in the presence of a time-invariant 
feedback. We provided a sequence of inner and outer bounds, 
and for some large families of channels we characterize the 
capacity region in terms of a 'multi-letter' expression, which is 
a first step toward deriving useful concepts in communication. 
For instance, we use this characterization to show that for 
a stationary and ergodic Markovian channel, the capacity is 
zero if and only if the capacity with feedback is zero. Further, 



we identify FS-MACs for which feedback does not enlarge 
the capacity region and for which source-channel separation 
holds. 

One future direction is to use the characterizations devel- 
oped in this paper to explicitly compute the capacity regions 
of classes of MACs with memory and feedback (other than 
the additive mod- channel), and to find optimal coding 
schemes. 
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